2 research outputs found
Reverse Knowledge Distillation: Training a Large Model using a Small One for Retinal Image Matching on Limited Data
Retinal image matching plays a crucial role in monitoring disease progression
and treatment response. However, datasets with matched keypoints between
temporally separated pairs of images are not available in abundance to train
transformer-based model. We propose a novel approach based on reverse knowledge
distillation to train large models with limited data while preventing
overfitting. Firstly, we propose architectural modifications to a CNN-based
semi-supervised method called SuperRetina that help us improve its results on a
publicly available dataset. Then, we train a computationally heavier model
based on a vision transformer encoder using the lighter CNN-based model, which
is counter-intuitive in the field knowledge-distillation research where
training lighter models based on heavier ones is the norm. Surprisingly, such
reverse knowledge distillation improves generalization even further. Our
experiments suggest that high-dimensional fitting in representation space may
prevent overfitting unlike training directly to match the final output. We also
provide a public dataset with annotations for retinal image keypoint detection
and matching to help the research community develop algorithms for retinal
image applications
Improving Mitosis Detection Via UNet-based Adversarial Domain Homogenizer
The effective localization of mitosis is a critical precursory task for
deciding tumor prognosis and grade. Automated mitosis detection through deep
learning-oriented image analysis often fails on unseen patient data due to
inherent domain biases. This paper proposes a domain homogenizer for mitosis
detection that attempts to alleviate domain differences in histology images via
adversarial reconstruction of input images. The proposed homogenizer is based
on a U-Net architecture and can effectively reduce domain differences commonly
seen with histology imaging data. We demonstrate our domain homogenizer's
effectiveness by observing the reduction in domain differences between the
preprocessed images. Using this homogenizer, along with a subsequent retina-net
object detector, we were able to outperform the baselines of the 2021 MIDOG
challenge in terms of average precision of the detected mitotic figures